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GLOBAL AND LOCAL STATISTICS CONTROLLED NOISE REDUCTION 

SYSTEM 

By 

Peng Lin 
and 

Yeong-Taeg Kim 



Field of the Invention 

The present invention relates generally to video image 
10 processing, and more particularly to noise reduction in 
video images. 

Background of the Invention 

Video images are often corrupted by noise during the 
video image acquisition or transmission process. In order 

15 to improve the visual appearance of such images , an 

effective filtering technique is often required to reduce 
the noise level therein. Traditional noise reduction 
techniques mainly involve applying a linear filter such as 
an averaging filter to all of the pixels in a video frame 

20 ("image") . While this reduces noise level in the image, 
such a linear filtering technique also indiscriminately 
blurs edges in the image. 
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In order to prevent image edge blurring, a noise 
reduction filter must be adaptive to local structures, such 
as edges, in the image. One such adaptive technique is 
known as directional filtering. Directional filtering 
5 attempts to avoid image blurring by adapting linear 

filtering to image edge directions in such a way that the 
filter utilized is always applied along the edge direction 
not across the edge direction. 

10 FIG.l shows a block diagram of an example directional 

filter 100. At each image pixel, first the 2-D local 
variance is computed by a local variance calculator 120 for 
a small window. Then, the 1-D local variances are computed 
along the horizontal, vertical, diagonal from upper left to 

15 lower right, and diagonal from upper right to lower left 
directions within the same window of pixels. To determine 
the edge direction, the 2-D variance is compared with a 
predetermined threshold in an edge direction detector block 
140. If the 2-D variance is less than the threshold, then 

20 no edge is present at the pixel, and the pixel is 

considered having "no direction" . If the 2-D variance is 
greater than the threshold, then an edge is present at that 
pixel, and the direction with the smallest 1-D variance is 
considered as the edge direction of the pixel. Utilizing a 
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filter 160, at "no direction" pixels, a 2-D average filter 
is applied. At a pixel with a detected edge direction, a 
1-D average filter is applied along the detected direction. 
By filtering along image edge directions, the directional 
5 filter 100 is able to retain most of the image structures 
while reducing the noise level of the input image. 

There are two major shortcomings to the directional 
filtering technique. The first is that the threshold value 
must be manually tuned and usually it is difficult to 
select the right value. An improperly selected threshold 
value will cause either image blurring or insufficient 
noise reduction. The second shortcoming of the directional 
filter is that the filter strength is fixed. That means a 
relatively clean image is processed the same way as a 
highly noisy image. This causes the relatively clean image 
to unnecessarily lose some fine structures and be degraded. 

Brief Summary of the Invention 

20 The present invention addresses the above 

shortcomings. As such, in one embodiment the present 
invention provides a global and local statistics controlled 
noise reduction system wherein the video image noise 
reduction processing is effectively adaptive to both image 
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local structure and global noise level. And, a noise 
estimation method according to the present invention 
provides reliable global noise statistics to the noise 
reduction system. 

5 

Such a global and local statistics controlled noise 
reduction system dynamically/adaptively configures a local 
filter for processing each image pixel, and processes the 
pixel with that local filter. The filtering process of the 

10 noise reduction system is controlled by both global and 
local image statistics that are also computed by the 
system. In one example, the local statistics computed by 
the system are 1-D and 2-D local variances, and the global 
statistics computed by the system is the global noise 

15 standard deviation. The local filter configured by the 
system for each image pixel has different filtering 
directions and variable strength for different pixels. The 
direction of the local filter is determined by 1-D local 
variances. The strength of the local filter is computed 

20 directly from the local variances and the global noise 
standard deviation. 

According to a further aspect of the present 
invention, the global noise standard deviation is estimated 
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by a noise estimation method. First, the image is divided 
into overlapping or non-overlapping blocks, and the mean 
and the standard deviation of each block are calculated. 
Then, the smallest standard deviation is found together 

5 with the corresponding block mean. After the smallest 
standard deviation and its corresponding mean have been 
found, a "saturation checking'' process is applied to 
determine whether the block with the smallest standard 
deviation has saturated. This determination is based on 

10 the relation between the smallest standard deviation and 
its corresponding mean. 



If saturation is not detected, the calculated block 
standard deviations that are within a neighboring interval 
15 centered at the smallest standard deviation are averaged, 
and the average value is taken as the estimated global 
noise standard deviation. The radius of the neighboring 
interval depends on the value of the smallest standard 
deviation. 

20 

If saturation is detected, first a saturation 
compensation term is added to the smallest standard 
deviation to generate a compensated smallest standard 
deviation. The saturation compensation term is computed 
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from the smallest standard deviation and its corresponding 
mean. Then, the calculated block standard deviations that 
are within a neighboring interval centered at the 
compensated smallest standard deviation are averaged to 
5 obtain the estimated global noise standard deviation. The 
noise standard deviation is used in the noise reduction 
system. 

Other objects, features and advantages of the present 
10 invention will be apparent from the following specification 
taken in conjunction with the following drawings. 



Brief Description of the Drawings 

FIG. 1 shows a block diagram of a prior art 
15 directional filter; 

FIG. 2 shows a functional block diagram of an 
embodiment of a global and local statistics controlled 
noise reduction system according to the present invention; 

FIG. 3 illustrates example directions for computing 
20 the 1-D variances in the system of FIG. 2; 

FIG. 4 shows an example curve representing dependency 
of the filter strength on local variance and global noise 
standard deviation; 
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FIG. 5 shows a function block diagram of an embodiment 
of a noise estimation system utilized for the global 
statistics computing unit of FIG. 2; and 

FIGS. 6A and 6B show example diagrams illustrating the 
5 effect of pixel values driven into saturation (0 or 255) by 
noise . 

Detailed Description of the Invention 

As noted above, in one embodiment the present 
invention provides a global and local statistics controlled 

10 noise reduction system wherein the video image noise 

reduction processing is effectively adaptive to both image 
local structure and global noise level. And, a noise 
estimation method according to the present invention 
provides reliable global noise statistics to a noise 

15 reduction system. The system dynamically configures a 

local filter for processing each image pixel, and processes 
the pixel with that local filter. The filtering process is 
controlled by both global and local image statistics. In 
one example, the local statistics computed by the system 

20 are 1-D and 2-D local variances, and the global statistics 
computed by the system is the global noise standard 
deviation. The dynamically configured local filter has 
different directions and variable strength for different 
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pixels. The direction of the local filter is determined by 
1-D local variances. The strength of the local filter is 
computed directly from the local variances and the global 
noise standard deviation. 

5 

The global noise standard deviation is estimated by a 
preferred noise estimation method that comprises the 
following steps. First, the image is divided into 
overlapping or non-overlapping blocks, and the mean and the 

10 standard deviation of each block are calculated. Then, the 
smallest standard deviation is found together with the 
corresponding block mean. After the smallest standard 
deviation and its corresponding mean have been found, a 
"saturation checking" process is applied to determine 

15 whether the block with the smallest standard deviation has 
saturated. The process is based on the relation between 
the smallest standard deviation and its corresponding mean. 



If no saturation is detected, the calculated block 
20 standard deviations that are within a neighboring interval 
centered at the smallest standard deviation are averaged, 
and the average value is taken as the estimated global 
noise standard deviation. The radius of the neighboring 
interval depends on the value of the smallest standard 
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deviation. If saturation is detected, first a saturation 
compensation term is added to the smallest standard 
deviation to generate a compensated smallest standard 
deviation. The saturation compensation term is computed 

5 from the smallest standard deviation and its corresponding 
mean. Then, the calculated block standard deviations that 
are within a neighboring interval centered at the 
compensated smallest standard deviation are averaged to 
obtain the estimated global noise standard deviation (the 

10 radius of the neighboring interval depends on the value of 
the compensated smallest standard deviation) . The global 
noise standard deviation is used in the noise reduction 
system. 



15 An example of the noise reduction system according to 

the present invention is now described. FIG. 2 shows a 
functional block diagram of the example global and local 
statistics controlled noise reduction system 200 according 
to an embodiment of the present invention. The system 200 

20 comprises a Global Statistics unit 210, a Local Statistics 
unit 220, a Direction Detector 230, a Filter Generator 240, 
and a Pixel Filtering unit 250. A digital video input 
image is first supplied to both the Global Statistics unit 
210 and the Local Statistics unit 220. The Global 
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Statistics unit 210 estimates the global noise statistics 
using said noise estimation method (also described further 
below) . The output of the Global Statistics unit 210 is 
the global noise standard deviation a , which is supplied to 
5 the Filter Generator 240. The Local Statistics unit 220 
computes the 2-D local variance within a small window 
centered at the current pixel and the 1-D local variances 
along four directions within the same window, 

10 FIG. 3 illustrates an example two-dimensional window 

300 including nine image pixels 310, and example directions 
for computing the 1-D local variances for pixel (i 9 j), where 
/and yare the indices for pixel row and column, 
respectively. The designations Li, L 2 , L 3 and L 4 denote the 

15 horizontal, vertical, diagonal from upper left to lower 
right, and diagonal from upper right to lower left 
directions, respectively. Further, the designation g\ 
denotes the 1-D local variance computed for direction L*, 
wherein k = 1, 2, 3 or 4 . And, the designation cr 0 2 denotes 

20 the 2-D local variance. 

In the example of FIG. 3, P(iJ) denotes e.g. the image 
gray-scale value of the pixel at position (i 9 j), wherein the 
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local variances a] {k = 0, 1, 2, 3, 4) at pixel (/,/) are 
computed within the 3x3 window 300 by example as follows. 



If k = 0, then cr 0 2 is the 2-D variance, and is computed 
by: 

<7o=(ZZ(^('' + s>/ + 0-A,) 2 )/9, 
where // 0 is the corresponding 2-D mean, defined as: 

^o=(ES /> ( i '+^-/ + o)/9. 

If Jr > 0, then cr* (k = 1, 2, 3, 4) are 1-D variances 
along the direction L k (k = 1, 2, 3, 4), and are 
computed by: 

c\ = (W, J ~ 1) " M l f + » " ^ ) 2 + W, J + l)- A f ) 1 3 ; 

<t 2 2 = ((/>(/ - 1, j) - n 2 f + (P(i, y) - // 2 ) 2 + (P(/ + 1, j) - m 2 ) 2 ) / 3 ; 

a] = ((/>(/ -1,7-1)- & ) 2 + (/>(/, j) - M 3 ) 2 + {P(f + 1, J + 1) " V,) 2 ) 1 3 ; 

cr 4 2 = ((/>(/ - 1, y + 1) - ^ ) 2 + (/»(/, y) - // 4 ) 2 + (P(/ + 1, y - 1) - // 4 ) 2 ) / 3 ; 

where (£" = 1, 2, 3, 4) are the means along the 
direction L k (k = 1, 2, 3, 4), and are computed by: 

M\ = w. y - 1) + y) + y + 1» / 3 ; 

fit = (P(i - 1, j) + P(i, j) + pq + 1, y» / 3 ; 
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M3 = (/>(/ - 1, y - 1) + P(i, j) + P(i + 1,7 + 1)) / 3 ; 
// 4 =(P(i-l,y + l) + / , (/,y) + / l (i + W-l))/3 - 

After computing the local variances, the Local 
5 Statistics unit 220 (FIG. 2) provides the 1-D local 

variances to the Direction Detector 230 to determine the 
local edge direction. The Direction Detector 230 then 
selects the direction that has the smallest 1-D variance as 
the local edge direction, and provides it to the Filter 
10 Generator 240. The Local Statistics unit 220 also provides 
the computed 1-D and 2-D local variances to the Filter 
Generator 240, to generate/configure a local filter based 
on the statistics quantities provided by both the Local 
Statistics unit 220 and the Global Statistics unit 210. 

15 

The Filter Generator 240 generates a local filter for 
the pixel to be filtered. The direction of the local 
filter is the local edge direction detected by the 
Direction Detector 230. The strength of the local filter 
20 is computed by using the global noise standard deviation 
aprovided by the Global Statistics unit 210 and the local 
variances a\ (k = 0, 1, 2, 3, 4) provided by the Local 
Statistics unit 220. For edge direction L k (k = 1, 2, 3, 
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4), the designation a k (k = 1, 2, 3, 4) denotes the 
corresponding filter strength along those directions. 
Further, the designation a 0 denotes the filter strength for 
non-edge area filtering. While a k {k = 1, 2, 3, 4) 
5 controls the strength for filtering along the edge 
direction, a 0 controls the strength for non-edge area 
filtering. 

The filter strengths a k (k = 1, 2, 3, 4) for edge 
direction L k (k = 1, 2, 3, 4) are functions of the global 
10 noise standard deviation and the local variance, and are 
computed in one example as: 

a k = min(2cr,max(3cr -0^,0)) /(2cr) ; 

wherein min(a,&) is the minimal function that returns 
the smaller one of the two values a and b, and max(a,&) is 
15 the maximal function that returns the larger one of the two 
values a and b . 



FIG. 4 shows an example curve/plot 400 of the filter 
strength function^. When <j k (i.e., the square root of the 
20 local variance) is small in comparison with the global 
noise standard deviation a (indicating that the local 
change is caused by noise) , the local filter has full 
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strength. When o^is large in comparison to a (indicating 
that the local change along the detected direction is 
caused by image structure), the local filter has zero 
strength. In between, the local filter strength 
5 continuously varies with a k . Further, the filter strength 
a k varies with the global noise standard deviation a (i.e., 
it increases as the global noise standard deviation 
increases) . 

10 The filter strength a Q f or non-edge area is computed 

similarly as following: 

a 0 = min(2cr,max(3cr -<r 0? 0))/(2cr) . 

The curve for a Q is similar to that for a k {k = 1, 2, 

3, or 4) shown in FIG. 4. 

15 Using the detected local edge direction L k (k = 1, 2, 

3, or 4), the edge direction filter strengths a k (k = 1, 2, 
3, or 4), and the non-edge area filter strength a 0 computed 
above, the Filter Generator unit 240 (FIG. 2) 
generates/configures the local filter f k {k = 1, 2, 3, or 4) 

20 as follows. 



15 



If the detected direction is Li, then /, is a 2-D local 
filter for horizontal direction, and is defined as: 



a* 



a 0 + 3a, (1 - a Q ) a 0 + 3(3 - 2a,)(l -a 0 ) a o + 3 «i(! - a o) 



If the detected direction is L 2 , then f 2 is a 2-D local 
filter for vertical direction, and is defined as: 



ji 9 



a 0 a 0 +3a 2 (\-a Q ) a 0 
a 0 a 0 +3(3-2a 2 )(l-a 0 ) a 0 
a Q a 0 +3a 2 (\-a 0 ) a 0 



If the detected direction is L 3 , then / 3 is a 2-D local 



filter for the diagonal direction from upper left to 
lower right, and is defined as: 



i 



a 0 + 3a 3 (l-a 0 ) 



a 0 +3(3-2a 3 )(l-a 0 ) 



a 0 +3a 3 (l-a 0 ) 



and 



If the detected direction is L 4 , then / 4 is a 2-D local 
filter for the diagonal direction from upper right to 
lower left, and is defined as: 
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'"5 



a 0 +3a 4 (l-tf 0 )~ 



a 0 +3(3-2a 4 )(l-a 0 ) 



a 0 +3tf 4 (l-a 0 ) 



The generated local filter f k is supplied to the Pixel 
Filtering unit 250 (FIG. 2). The pixel is then filtered 
5 using the weighted sum of its neighboring pixels within a 
e.g. 3x3 window with the corresponding filter coefficients 
as the weights. 



As noted, the Global Statistics unit 210 estimates the 
10 global noise statistics using a preferred noise estimation 
method. Different methods have been proposed to estimate 
the noise present in the images, such as those described in 
the paper by S. I. Olsen: "Noise Variance Estimation in 
Images: An Evaluation", Graphical Models and Image 
15 Processing, vol. 55, no. 4, pp. 319-323, 1993. 



However, existing methods have not properly considered 
saturation effects (i.e., pixel values driven into 
saturation (0 or 255) by the noise, causing inaccurate 
20 estimates (in most cases underestimates) of noise) . 

Inaccurate noise estimate can have serious impact on the 
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performance of the noise reduction system that is 
controlled by global noise statistics. 

FIG . 5 shows a functional block diagram of an example 
5 noise estimation system 500 according to the present 

invention, for the Global Statistics Computing unit 210 of 
FIG. 2. The example noise estimation system 500 according 
to the present invention is capable of handling said 
saturation effect and providing accurate noise estimates. 
10 The input image is first divided into overlapping or non- 
overlapping blocks B n of size HxW; (n = 1,2,..., N) ; where 
N is the total number of blocks, and the mean and the 
standard deviation of each block are computed by the Mean 
and Standard Deviation Calculator unit 510. The selected 
15 block size should not be too small to ensure a robust 

estimate. Preferably, the block size is e.g. 7x7 or 5x9 
pixels (other block sizes can be used) . The mean m n and the 
standard deviation d n of block B n are computed in the unit 
510, respectively, as: 



20 



HxW 
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The computed block standard deviations and means are 
then provided to a Minimal Finder 520. The block standard 
deviations are also provided to a Selective Averaging unit 
530. 

5 

The Minimal Finder 520 finds the smallest standard 
deviation, and records the smallest standard deviation and 
its corresponding block mean as d 0 and m 0 , respectively, 
wherein the values d Q and m 0 are then supplied to a 
10 Saturation Checker 540. 

The Saturation Checker 540 checks whether saturation 
has occurred in the block with the smallest standard 
deviation d 0 . That is, the Saturation Block checker 
15 determines/detects if pixel values in the block are driven 
into saturation (e.g., 0 or 255) due to noise, which may 
cause inaccurate estimates of image noise. If saturation 
has occurred, a Saturation Compensator 550 compensates for 
d 0 . 

20 

Examples of saturation detection criteria and the 
compensation methods are provided below in conjunction with 
FIGS. 6A-6B which illustrate examples of the effect of 
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pixel values driven into saturation by noise (e.g., pixel 
value is at a lower limit such as 0 or at an upper limit 
such as 255) . 

For this example an upper limit UL = 255, a lower 
limit LL = 0 and a mid value M = 128. Such that, if 
jw 0 <128 and d 0 >m Q -0, then saturation is detected at 

the lower limit 0 (FIG. 6A) . In this case d 0 is 

compensated as d 0 = d 0 + K-(d 0 -(m 0 -0)) , wherein d 0 denotes 
the compensated smallest standard deviation; 
If m 0 >\2$ and d 0 >255-m 0 , then saturation is detected 
at the upper limit 255 (FIG. 6B) . In this case d 0 is 
compensated as d Q =d 0 +K-(d 0 -(255-w 0 )) ; 

Otherwise, no saturation is detected. In this case no 

• — * 

compensation is needed for d 0 , therefore d 0 =d 0 . 

In the above expressions for d 0 , the compensation 
parameter^ is empirically determined. Preferably in this 
example £ = 5.0 is used. As those skilled in the art will 
recognize, the saturation detection method can be easily 
generalized to other situations where the images are 
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represented by different bit values and therefore have 
different values for UL, LL and M. 

The compensated smallest standard deviation d 0 is then 

supplied to the Selective Averaging unit 530 (FIG, 5) to 
generate the final estimate of the global noise standard 
deviation a , utilized in the system 200 of FIG. 2. 

The Selective Averaging unit 530 (FIG. 5) first 
selects those block standard deviations (provided by the 
Mean and Standard Deviation Calculator 510) that are within 
a selected range of (e.g., close to) the compensated 

smallest standard deviation d 0 . In one example, a block 

standard deviation d n (n = 1,2, ... ,N) is considered within 

a range of d 0 if | d n -d Q |< max(rf 0 ,l) . The selected block 
standard deviations are then averaged, and the average 
value is taken as the final estimate of the global noise 
standard deviation a, for use in the system 200 (FIG. 2). 

While this invention is susceptible of embodiments in 
many different forms, there are shown in the drawings and 
will herein be described in detail, preferred embodiments of 
the invention with the understanding that the present 
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disclosure is to be considered as an exemplification of the 
principles of the invention and is not intended to limit the 
broad aspects of the invention to the embodiments 
illustrated. The aforementioned systems 200 and 500 

5 according to the present invention can be implemented in 
many ways, such as program instructions for execution by a 
processor, as logic circuits, as ASIC, as firmware, etc., as 
is known to those skilled in the art. Therefore, the 
present invention is not limited to the example embodiments 

10 described herein. 



The present invention has been described in 
considerable detail with reference to certain preferred 
versions thereof; however, other versions are possible. 
15 Therefore, the spirit and scope of the appended claims 

should not be limited to the description of the preferred 
versions contained herein. 
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